Skip to content

HTTP server that exposes stores, arrays, groups#3732

Open
d-v-b wants to merge 8 commits intozarr-developers:mainfrom
d-v-b:feat/experimental-server
Open

HTTP server that exposes stores, arrays, groups#3732
d-v-b wants to merge 8 commits intozarr-developers:mainfrom
d-v-b:feat/experimental-server

Conversation

@d-v-b
Copy link
Contributor

@d-v-b d-v-b commented Feb 28, 2026 •
edited
Loading

This PR adds an experimental http server in experimental.serve. This server can expose stores over http. It can also expose arrays and groups over http. Exposing a store means exposing the entire key: value space of the store. Exposing an array means only exposing the metadata + chunks. Exposing a group means only exposing sub-groups and sub-arrays. See #3731 for more on this distinction.

The server is an optional dependency implemented via starlette. It handles byte-range reads and other http methods. CORS headers and allowed methods can be configured. I'm considering handling prefix requests like foo/bar by returning a simple HTML document that lists the visible keys under foo/bar/, for user-friendliness and to aid httpstore readers that use such responses for listing contents. I'd also like to implement convenience functions for kicking off a server from jupyter and a CLI.

Opening as a draft while I work on this.

@github-actions github-actions bot added the needs release notes Automatically applied to PRs which haven't added release notes label Feb 28, 2026
@d-v-b d-v-b marked this pull request as ready for review March 1, 2026 20:10
d-v-b
d-v-b commented Mar 1, 2026
View reviewed changes
tests/test_examples.py
@@ -19,47 +19,27 @@
ZARR_PROJECT_PATH = Path(".").absolute()
Copy link
Contributor Author

@d-v-b d-v-b Mar 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

changes in this file are simplifications to our examples testing infrastructure. Instead of re-writing the script header, we just override the declared zarr dep in the invocation of uv run ...

@github-actions github-actions bot removed the needs release notes Automatically applied to PRs which haven't added release notes label Mar 1, 2026
d-v-b
d-v-b commented Mar 1, 2026
View reviewed changes
src/zarr/core/keys.py
@@ -0,0 +1,178 @@
"""Utilities for determining the set of valid store keys for zarr nodes.
Copy link
Contributor Author

@d-v-b d-v-b Mar 1, 2026

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

we eventually need to find a more natural place for this code. I'm not sure which module it should live in.

@d-v-b d-v-b requested a review from maxrjones March 1, 2026 20:29
@d-v-b
Copy link
Contributor Author

d-v-b commented Mar 1, 2026

I'm considering handling prefix requests like foo/bar by returning a simple HTML document that lists the visible keys under foo/bar/, for user-friendliness and to aid httpstore readers that use such responses for listing contents. I'd also like to implement convenience functions for kicking off a server from jupyter and a CLI.

Didn't end up doing these things. We can add them later if people are interested.

@d-v-b d-v-b requested a review from a team March 2, 2026 00:17
@d-v-b
Copy link
Contributor Author

d-v-b commented Mar 2, 2026 •
edited
Loading

here's a demo of this functionality:

# /// script
# requires-python = ">=3.11"
# dependencies = [
#   "zarr[server] @ git+https://github.com/d-v-b/zarr-python@feat/experimental-server",
# ]
# ///

import zarr
import numpy as np
from zarr.storage import MemoryStore
from zarr.experimental.serve import serve_node

data = np.arange(1000, dtype='uint8').reshape(10, 10, 10)
store = MemoryStore({})
z = zarr.create_array(store, data=data, write_data=True)
serve_node(
    z,
    host="127.0.0.1",
    port=8000,
    cors_options={"allow_origins": ["*"], "allow_methods":["GET", "HEAD"]}
    )

If you run that script, then visit https://neuroglancer-demo.appspot.com/#!%7B%22dimensions%22:%7B%22dim_0%22:%5B1%2C%22%22%5D%2C%22dim_1%22:%5B1%2C%22%22%5D%2C%22dim_2%22:%5B1%2C%22%22%5D%7D%2C%22position%22:%5B1.5%2C4.5%2C5.5%5D%2C%22crossSectionScale%22:0.03421811831166599%2C%22projectionOrientation%22:%5B-0.06506630778312683%2C-0.14447830617427826%2C-0.003086749231442809%2C0.9873615503311157%5D%2C%22projectionScale%22:53.328548429788874%2C%22layers%22:%5B%7B%22type%22:%22image%22%2C%22source%22:%22http://127.0.0.1:8000/%7Czarr3:%22%2C%22tab%22:%22source%22%2C%22name%22:%228000%22%7D%5D%2C%22selectedLayer%22:%7B%22visible%22:true%2C%22layer%22:%228000%22%7D%2C%22layout%22:%224panel-alt%22%7D, you should see the array data

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@maxrjones maxrjones Awaiting requested review from maxrjones

At least 1 approving review is required to merge this pull request.

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@d-v-b